Complex entities, such as large systems and large networks, that ultimately must function with other equipment depend upon various standards to ensure that the entities will function when installed. Before installation of a system or a network, it is very valuable to uncover as many errors as possible in the implementation. This reduces the costly process of identifying and correcting errors after the system or network is installed with other equipment. Further, in those situations where the manufacturer and the system and network owner are different from each other, it is very common to require conformance to a specific standard of operation. Thus, thorough testing of complex entities is desirable from both a cost standpoint and a standards standpoint.
Complex systems and complex networks understandably have complex standards that specify the operation they must provide in order to be in conformance thereto. Moreover, exhaustive testing of some complex systems and networks, i.e. testing of every possible combination of states, is impractical if not impossible. Instead, testing each state of a complex system or network at least once is a commonly used strategy. This type of testing is called conformance testing because it tests the responses of the entity against the responses required to conform to the governing standard.
Conformance testing for some moderately complex entities such as systems utilizing an ISDN protocol or a VLSI circuit are disclosed in U.S. Pat. Nos. 4,692,921 issued to A. Dahbura et al. on Sep. 8, 1987, and U.S. Pat. No. 4,991,176 issued to Dahbura et al. on Feb. 5, 1991. Both of these patents are commonly assigned to the assignee hereof and both are hereby incorporated by reference. The former patent presents a method for generating unique input/output sequences of optimum length that test every edge of a finite state machine that represents the entity under test. The latter patent, is similar, but its method generates unique input/output sequences of optimum cost instead of optimum length. Neither of these known methods for generating test sequences has a broad enough scope to provide all the capabilities for generating and storing test sequences for conformance testing of more complex entities, such as networks governed by the OSI Network Management protocol and systems.
Thus, there is a need for a conformance test method and apparatus that provides the generation, selection, parametrization and execution of test cases to test complex entities, such as an OSI Network Management protocol implementation and the system hardware that the OSI Network Management protocol is implemented on.